Computer graphics systems which are commonly used today generally provide a representation of the workspace, or display screen, occupied by the various elements of the scene. Designing a graphical user interface (GUI) for viewing and manipulating a virtual three-dimensional (3D) space requires creating a set of objects and their properties, arranging them in a scene, setting a viewing specification, determining lighting and rendering parameters, and deciding how to update these decisions for each frame. Some of these decisions may be fully constrained; for example, a simulation may determine the position and shape of certain objects, or the viewing specification may be explicitly controlled by the user. In contrast, other decisions must be resolved by the GUI designer. Of particular interest are those decisions that determine the spatial layout of the projections of objects on the view plane. These decisions can be referred to collectively as view management. For example, some objects may be sufficiently important to the user's task that they should not be occluded. In addition, the members of a group of related objects may need to be placed together to emphasize their relationship.
In a static scene, observed from a fixed viewing specification, view-management decisions might be made in advance, by hand, and hold throughout the life of an application. It is also common in both 2D and 3D interactive GUIs to avoid automating view management when possible. For example, a fixed area of the screen may be dedicated to a menu, or the user may explicitly control the positions of permanent menus or temporary pop-up menus, or the positions and sizes of windows. However, hard-wired or direct-manipulation control becomes problematic when applied to dynamic scenes that include autonomous objects and to head-tracked displays. In this type of situation, continual and unpredictable changes in object geometry or viewing specification result in continual changes in the spatial and visibility relationships among the projections on the view plane. In these cases, view-management decisions must be made on the fly if they are to take dynamic changes into account.
Augmented reality applications are especially challenging in this regard. Virtual and physical objects reside in the same 3D space and there may be no way to control the behavior of many of the physical objects. For example, the view through an optical see-through head-worn display includes all the physical objects that occupy the user's field of view in addition to the virtual objects being displayed. In this case, the portion of the field of view that can be augmented may be relatively small.
It would be desirable to manage the virtual space such that objects could be added or moved in a controlled manner. For example, it would be desirable if virtual annotations could be added to the virtual space and interspersed among the objects they describe and reconfigured automatically and understandably to take into account changes in the objects themselves and how they are viewed.